Visually-impaired-accessible building safety system

ABSTRACT

Building safety systems, methods, and mediums are provided. A method includes receiving a voice input by the building safety system. The method includes receiving voice data produced by a speech recognition process performed on the voice input. The method includes determining a response to the voice input based on the voice data. The method includes producing the response by the building safety system.

RELATED APPLICATIONS

This application is a continuation of application Ser. No. 15/265,546,filed Sep. 14, 2016, now granted as U.S. Pat. No. 9,934,658, issued Apr.3, 2018, and claims priority to PCT/US2017/051246, filed Sep. 13, 2017.

TECHNICAL FIELD

The present disclosure is directed, in general, to building safety andautomation systems and, more particularly, to building safety systemsthat accommodate the visually impaired and other people.

BACKGROUND OF THE DISCLOSURE

Building automation systems encompass a wide variety of systems that aidin the monitoring and control of various aspects of building operation.Building automation systems include security systems, fire safetysystems, lighting systems, and HVAC systems. The elements of a buildingautomation system are widely dispersed throughout a facility. Thesebuilding automation systems typically may have one or more centralizedcontrol stations from which system data may be monitored and variousaspects of system operation may be controlled and/or monitored. Abuilding can have one or more fire safety control panels as part of thefire safety system of the building.

To allow for monitoring and control of the dispersed control systemelements, building automation systems often employ multi-levelcommunication networks to communicate operational and/or alarminformation between operating elements, such as sensors (such as smokedetectors) and actuators (such as strobes or other notificationappliances), a fire safety control panel, and a centralized monitoringsystem.

Fire safety systems can play an integral part in building automationsystems. A typical fire safety system uses automated smoke and heatdetectors, and relies on manual “pull stations” for individuals toactivate the fire safety system. Pull stations are generally required tohave braille markings for the use of blind person, but if those personsare not familiar with a building, it can be very difficult for avisually-impaired individual to locate a pull station.

Moreover, if smoke develops in area of a building or residence with adisarmed or non-functioning detector, a person who is not blind may havedifficulty locating and activating a pull station to cause the firesafety system to notify an emergency responder.

Furthermore, other non-fire emergency events (such as a terrorist in abuilding) may cause a person to want to locate a pull station to signalthe fire safety system or building automation system to notify anemergency responder.

Accordingly, there is a need for an improvement in pull stations orother devices in a fire safety system or building automation system toenable a visually-impaired person to signal such system of an emergencyevent in order to notify an emergency responder.

SUMMARY OF THE DISCLOSURE

Various disclosed embodiments relate to systems and methods for aspeech-recognition-enabled building safety system (which may be a firesafety system, security system, a combination thereof or other subsystemof a building automation system). A method includes receiving a voiceinput by the building safety system. The method includes receiving voicedata produced by a speech recognition process performed on the voiceinput. The method includes determining a response to the voice inputbased on the voice data. The method includes producing the response bythe building safety system.

A building safety system includes a safety control panel and a pluralityof building safety devices in communication with the safety controlpanel. The building safety system configured to receive a voice input,receive voice data produced by a speech recognition process performed onthe voice input, determine a response to the voice input based on thevoice data, and produce the response.

In various embodiments, the building management system also determineswhether the voice data corresponds to an authorized user and wherein thebuilding safety system produces the response when the voice data isdetermined to correspond to the authorized user. In various embodiments,the speech recognition process is performed by a mobile device, and thevoice data is transmitted to the building safety system. In variousembodiments, determining a response to the voice input includesdetermining whether the voice input is one of a request for directionsto a pull station, a distress sound, or a distress word. In variousembodiments, producing the response includes activating a notificationdevice controlled by the building safety system to signal an alarm afterdetermining the voice input is one of a request for directions to a pullstation, a distress sound or a distress word. In various embodiments,wherein the building safety system includes a plurality of buildingsafety devices each having a speaker, and after determining the voiceinput is one of a distress sound or a distress word, the building safetysystem produces an audible response via the speaker of a first of thebuilding safety devices, the audible response being a phrase to promptan individual to confirm an emergency event is occurring. In variousembodiments, the voice input is received by a building safety deviceimplemented in one of a pull station, a smoke detector, a heat detector,a notification device, an emergency light, an exit sign, or apublic-address speaker. In various embodiments, the response is producedby a building safety device implemented in one of a pull station, asmoke detector, a heat detector, a notification device, an emergencylight, an exit sign, or a public-address speaker. In variousembodiments, the building management system also determines a locationof an individual that transmitted the voice input relative to a buildingsafety device of the building safety system, and the response is audibledirections to the pull station or the exit relative to the location ofthe individual. In various embodiments, the building safety system has aplurality of building safety devices, each having a microphone, thevoice input is received by one of the building safety devices, and thespeech recognition process is performed by a safety control panel incommunication with the one building safety device. In variousembodiments, the building safety system has a plurality of buildingsafety devices in communication with a safety control panel; eachbuilding safety device having a microphone and a speech recognitioncircuit; and the speech recognition process is performed by one of thebuilding safety devices.

The foregoing has outlined rather broadly the features and technicaladvantages of the present disclosure so that those skilled in the artmay better understand the detailed description that follows. Additionalfeatures and advantages of the disclosure will be described hereinafterthat form the subject of the claims. Those of ordinary skill in the artwill appreciate that they may readily use the conception and thespecific embodiment disclosed as a basis for modifying or designingother structures for carrying out the same purposes of the presentdisclosure. Those skilled in the art will also realize that suchequivalent constructions do not depart from the spirit and scope of thedisclosure in its broadest form.

Before undertaking the DETAILED DESCRIPTION below, it may beadvantageous to set forth definitions of certain words or phrases usedthroughout this patent document: the terms “include” and “comprise,” aswell as derivatives thereof, mean inclusion without limitation; the term“or” is inclusive, meaning and/or; the phrases “associated with” and“associated therewith,” as well as derivatives thereof, may mean toinclude, be included within, interconnect with, contain, be containedwithin, connect to or with, couple to or with, be communicable with,cooperate with, interleave, juxtapose, be proximate to, be bound to orwith, have, have a property of, or the like; and the term “controller”means any device, system or part thereof that controls at least oneoperation, whether such a device is implemented in hardware, firmware,software or some combination of at least two of the same. It should benoted that the functionality associated with any particular controllermay be centralized or distributed, whether locally or remotely.Definitions for certain words and phrases are provided throughout thispatent document, and those of ordinary skill in the art will understandthat such definitions apply in many, if not most, instances to prior aswell as future uses of such defined words and phrases. While some termsmay include a wide variety of embodiments, the appended claims mayexpressly limit these terms to specific embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present disclosure, and theadvantages thereof, reference is now made to the following descriptionstaken in conjunction with the accompanying drawings, wherein likenumbers designate like objects, and in which:

FIG. 1 illustrates a block diagram of a building safety system in whichvarious embodiments are implemented;

FIG. 2 illustrates a block diagram of a data processing system in whichvarious embodiments of the present disclosure are implemented;

FIG. 3 illustrates a block diagram of a building safety device inaccordance with disclosed embodiments;

FIG. 4 illustrates a flowchart of a process in accordance with disclosedembodiments; and

FIG. 5 illustrates a flowchart of another process in accordance withdisclosed embodiments.

DETAILED DESCRIPTION

FIGS. 1 through 5, discussed below, and the various embodiments used todescribe the principles of the present disclosure in this patentdocument are by way of illustration only and should not be construed inany way to limit the scope of the disclosure. Those skilled in the artwill understand that the principles of the present disclosure may beimplemented in any suitably arranged device or system.

Embodiments of the present disclosure include systems and methods thataid visually-impaired persons, and others, to easily interact with abuilding safety system. In particular, disclosed embodiments employ uservoice recognition and other input techniques in fire safety devices(such as detection devices, emergency activation devices like pullstations, and alarm notification devices, like strobes or horns) orother building devices (such as public speakers, thermostats, andsecurity cameras) to receive input from a visually impaired person andto guide such person to an exit or to an emergency activation devicelike a fire alarm pull station to enable such person to signal thebuilding safety system to notify an emergency responder of the emergencyevent.

FIG. 1 illustrates a block diagram of a building safety system 100 inwhich various embodiments are implemented. In this illustrativeembodiment, the building safety system 100 includes a building controlsystem 103 that performs overall building management functions that caninclude safety controls, lighting controls, HVAC controls, and otherbuilding management functions. Building control system 103 can beimplemented by one or more date processing systems, as described herein,and any suitable controllers, sensors, actuators, and other devices toperform building management functions not discussed in detail herein.Building control system 103 may include a central or managementmonitoring system 102 and/or one or more display stations 106, each ofwhich can be implemented as a data processing system, mobile device, orother device configured to communicate with building control system 103.Each of the connections shown in the block diagram of FIG. 1 can includeany number of suitable connections, such as wired, wireless, or fiberoptic links, and can be implemented as a number of different types ofnetworks, such as, for example, the internet, a local area network(LAN), or a wide area network (WAN).

The building safety system 100 can include a fire safety control panel104, or several of them. Fire safety control panel 104 may beimplemented as a data processing system as described herein, using oneor more controllers and storage devices, or other components. Firesafety control panel 104 communicates with the various devices describedherein, and in specific embodiments is implemented as a fire controlpanel. Fire safety control panel 104 is configured to perform processesas described in more detail below.

Fire safety control panel 104 communicates with fire safety devices,including detection devices 110, notification devices 112, or pullstations 114. Detection devices 110 can include smoke detectors, heatdetectors, motion detectors, voice/sound detectors, cameras, infrared ormotion detectors, and other detection devices as described herein orknown to those of skill in the art. Notification devices 112 can includeloudspeakers, horns, sirens, strobes or other lights, display panels,signs, and other notification devices as described herein or known tothose of skill in the art, including remote annunciators and dialers tomake automated telephone calls. Pull stations 114 can include any meansof manually activating an emergency event, including mechanical firepull stations, panic buttons, or other emergency activation devices.

In various embodiments, the management monitoring system 102 can alsocommunicate with mobile devices 120, which may or may not be considereda part of the building safety system 100. For example, as described inmore detail below, the management monitoring system 102 (or detectiondevices 110) may include wireless access points or network connectionsthat can communicate with an individual's mobile device to performfunctions as described herein. Management monitoring system 102 cancommunicate with building devices 130, which can include (but are notlimited to) such devices as public speaker 131, thermostat 132, securitycamera 133, and security access readers/keypads 134. In accordance withembodiments disclosed herein, the management monitoring system 102 mayalso communicate with displays 135 and lights 136 that have wired orwireless network communication (including over power lines) with themanagement monitoring system.

According to disclosed embodiments, different types of devices areintegrated into a common unit. For example, a single unit may housenotification devices 112 such as a strobe light and loudspeaker, incombination with fire safety detection devices 110 such as heat andsmoke detectors as described herein. In various embodiments, the controllogic for various detection or notification functions is housed in thedevice itself, while in other embodiments, the device includes the inputor output device and the control logic is implemented by the fire safetycontrol panel 104.

For clarity in the description herein, a building safety device includesa fire safety device 110, 112, and 114 and/or other building device 131,132, 133, 134, 135 or 136 that has a voice activated emergency eventinterface 350 as further described in accordance with the embodimentsdisclosed herein.

FIG. 2 depicts a block diagram of a data processing system 200 in whichvarious embodiments are implemented. The data processing system 200 isan example of a system that can be used to implement building controlsystem 103 (or any management monitoring system 102 or display station106 thereof) or fire safety control panel 104. The data processingsystem 200 includes a processor 202 connected to a level twocache/bridge 204, which is connected in turn to a local system bus 206.The local system bus 206 may be, for example, a peripheral componentinterconnect (PCI) architecture bus. Also connected to local system busin the depicted example are a main memory 208 and a graphics adapter210. The graphics adapter 210 may be connected to a display 211.

Other peripherals, such as a local area network (LAN)/Wide AreaNetwork/Wireless (e.g. Wi-Fi) adapter 212, may also be connected tolocal system bus 206. An expansion bus interface 214 connects the localsystem bus 206 to an input/output (I/O) bus 216. The I/O bus 216 isconnected to a keyboard/mouse adapter 218, a disk controller 220, and anI/O adapter 222. The disk controller 220 may be connected to a storage226, which may be any suitable machine usable or machine readablestorage medium, including but not limited to nonvolatile, hard-codedtype mediums such as read only memories (ROMs) or erasable, electricallyprogrammable read only memories (EEPROMs), magnetic tape storage, anduser-recordable type mediums such as floppy disks, hard disk drives andcompact disk read only memories (CD-ROMs) or digital versatile disks(DVDs), and other known optical, electrical, or magnetic storagedevices.

Also connected to the I/O bus 216 in the example shown is an audioadapter 224, to which speakers (not shown) may be connected for playingsounds. The keyboard/mouse adapter 218 provides a connection for apointing device (not shown), such as a mouse, trackball, trackpointer,etc. In some embodiments, the data processing system 200 may beimplemented as a touch screen device, such as, for example, a tabletcomputer or touch screen panel. In these embodiments, elements of thekeyboard/mouse adapter 218 may be implemented in connection with thedisplay 211.

Those of ordinary skill in the art will appreciate that the hardwaredepicted in FIG. 2 may vary for particular implementations. For example,other peripheral devices, such as an optical disk drive and the like,also may be used in addition or in place of the hardware depicted. Thedepicted example is provided for the purpose of explanation only and isnot meant to imply architectural limitations with respect to the presentdisclosure.

One of various commercial operating systems, such as a version ofMicrosoft Windows™, a product of Microsoft Corporation located inRedmond, Wash. may be employed if suitably modified. The operatingsystem may modified or created in accordance with the present disclosureas described, for example, to implement virtual demand auditing of oneor more devices in a building.

LAN/WAN/Wireless adapter 212 may be connected to a network 235, such asfor example, communicating between the systems, panels, and devicesdescribed herein. As further explained below, the network 235 may be anypublic or private data processing system network or combination ofnetworks, as known to those of skill in the art, including the Internet.Data processing system 200 may communicate over network 235 to one ormore computers or mobile devices which are also not part of dataprocessing system 200, but may be implemented, for example, as aseparate data processing system 200.

According to disclosed embodiments, a voice activated emergency eventinterface 350 is employed in a building safety device 110, 112, 114,131, 132, 133, 134, 135 or 136 to provide a speech recognition(SR)-capable building safety device, such as an SR-capable pull station,that can be located and activated by a visually impaired person. Such anSR-capable building safety device can function to detect an emergencyevent based on voice recognition techniques described herein and toactivate the signaling of the emergency event to the fire safety controlpanel 104 or building control system 103, which in conventional firesafety systems required manual activation of a standard pull station.The SR-capable building safety device can be used throughout a buildingor common space, and can be activated by either voice command, asdescribed in more detail below, or by physical activation. As usedherein, an “alarm condition” refers to a condition detected as describedherein for which the building control system, fire safety control panel,or other building safety device provides a response such as sounding analarm, or notifying a responder or others within a building or outsidethe building via network 235.

The SR-capable building safety device as disclosed herein can help aperson locate the pull station or building safety device by: detectingthe person through the voice activated emergency event interface 350;determining the position of the detected person relative to the pullstation or building safety device via motion sensors employed in orintegrated with such pull station or building safety device; andproviding corresponding audio directions to such pull station orbuilding safety device.

FIG. 3 illustrates a block diagram of a building safety device 300 inaccordance with disclosed embodiments, that can be used to implement anSR-capable pull station or other SR-capable building safety device asdescribed herein. Note that not all of the elements described below arenecessary for every implementation, and that a similar device can beused in the building safety device for SR or notification purposeswithout being implemented as a pull station or including a mechanicalactivator.

Building safety device 300 can include a mechanical activator 302, suchas a level or switch, that operates as a conventional pull station tomanually indicate an alarm condition to the safety control panel.

Building safety device 300 can include a communication interface 314that communicates with the fire safety control panel 104 or otherdevices as described herein, to send and receive the signals asdescribed. Communication interface 314 can be an interface to a wired orwireless network, one or more analog or digital connections, a Bluetoothdevice, or otherwise. Communications interface 314 can also include suchfeatures as a Bluetooth beacon, Wi-Fi access point, global positioningsystem received, or others, and can thereby assist in providing locationservices to other devices. Communications interface 314 can operate in areal-time and continuous manner so that as the person is movingaccording to the directions of the system, the system is monitoring themovements of the person, and verbally correcting and directing theperson towards the pull station and then out of the building.

Standard pull stations typically have a mechanical activator 302 coupledto a discrete output wired connection to a fire safety control panel tosignal when the mechanical activator 302 is manually activated. However,such standard pull stations may be modified to include a voice activatedemergency interface 350 and a control circuit 352 to provide a buildingsafety device 300 as described in the disclosed embodiments herein. Inparticular, the control circuit 352 of a building safety device 300 mayinclude a controller 308, memory 309 for storing voice activatedemergency event logic 311, storage 310 that may store user voice samplesor patterns 313 a-313 n, I/O devices 333 and/or wireless transmitters334 for controlling the operation of the building safety device 300,storing any data, and performing functions as described herein. In someembodiments, control circuit 352 can also communicate with externalsystems, such as cloud-based systems, for monitoring, control, oranalysis purposes.

The voice activated emergency event interface 350 of the building safetydevice 300 may include a microphone 304, a voice recognition circuit331, and one or more location sensors or devices 330. The microphone 304is preferably tuned to detect audio from an individual at apredetermined distance from the building safety device 300 as installedin the building (i.e., within 500 feet) such that the individual is inaudio range of at least one building safety device 300 while in thebuilding. The voice recognition circuit 331 may include one or morenoise cancellation filters that may be selectively adjusted by atechnician to define background noise within proximity of the buildingsafety device 300 that is filtered out by the voice recognition circuit311 when in operation to enable the building safety device 300 toidentify the voice of an individual in accordance with the embodimentsdisclosed herein.

The location sensors or devices 330 may include an infrared detector orarray, a heat detector or array, or other detector that can detectconditions proximate to the building safety device 300 for determiningthe presence of an individual and to derive the direction of movement ofthe individual relative to the building safety device 300. In otherembodiments, the location sensors or devices 330 may also include acamera 317 that can capture images of an individual proximate to thebuilding safety device 300 for deriving the presence and direction ofmovement of such individual relative to the building safety device 300.In further embodiments, the location sensors or devices may includeanother standard motion sensor 320 that may be controlled via controller308 to detect the presence of and direction of movement of an individualwithin a distance of the building safety device 300 in which themicrophone 304 can detect the voice of the individual. For example, invarious embodiments, location sensors/devices 330 can include devicessuch as an ultrasonic ranging module, a sonic rangefinder, laserrangefinder, radar distance measurement, pulse repetition frequencydevices, GPS devices, or others, which can be used to help the detect aperson and provide the ability for the system to provide directions tothe individual.

As previously noted, the voice activated emergency event interface 350having the microphone 304, voice recognition circuit 331 and locationsensors/devices 330 can be installed in any fire safety device 110, 112,and 114 and/or other building device 131, 132, 133, and 134 to implementa building safety device 300 in accordance with the disclosedembodiments. In one embodiment, the controller 308 may send any audiodetected by the voice recognition circuit 331 of a building safetydevice 300 to the emergency safety application 228 for processing torecognize an individual identifying an emergency event and provide audioor visual notification feedback or guidance to the individual vianotification devices 112, displays 135, or light 136 in proximity to thebuilding device 300 that detected the audio from the individual. Inother embodiments, the voice activated emergency event interface 350 ofthe building safety device 300 may include a speaker 306 for providingaudio notification feedback or guidance generated by the controller 308when processing outputs from the voice recognition circuit 331 orreceived via the communication interface 314 from the emergency safetyapplication 228. Speaker 306 can be a loudspeaker, siren, or otheraudible notification device.

Building safety device 300 can also include other notification devicessuch as a display/light 319 to provide direct visual notifications toindividuals. For example, display/light 319 can be an LCD display, astrobe light, or other visual notification device. Note that while audioor visual notification are referred to in the alternative herein, theywill often be employed at the same time, and audio notification can bepreferred in implementations that are used to assist thevisually-impaired.

The various components can all be controlled by the controller 308,which can communicate with the safety control panel or other devicesover communication interface 314.

While in specific implementations, the building safety device 300 isimplemented as an SR-capable pull station, in other embodiments, thebuilding safety device 300 is implemented as a stand-alone safetysystem, without a mechanical pull-station function, to operate asdescribed herein. The speech-recognition functions can be performed inthe building safety device itself, in some embodiments, or the audiodata can be transmitted to the building management system or safetycontrol panel for processing and speech recognition.

According to disclosed embodiments, a building safety device 300 canperform speech-recognition functions by “listening” for specific wordsor voices using the microphone and the voice recognition circuit. Whenthe words or voices are detected, the building safety device 300 aloneor in cooperation with the emergency safety application 228 of thebuilding safety system 100 can respond with assistance, activate analarm condition, or otherwise, as described herein. For example, thebuilding safety device 300 may detect a person stating “Fire locate pullstation” and respond with an audio or visual indication of where thenearest fire safety device or pull station is located. In variousembodiments, the nearest fire safety device 300 may not be the same asthe nearest pull station, so if a first fire safety device “hears” suchkey words, the individual may be directed to a different fire safetydevice or pull station, such as by audibly sound, “a pull station islocated 15 feet down the hallway to the right. An exit is 20 feet downthe hallway to your right.”

In various embodiments, voice samples or patterns 313 a-313 n forauthorized individuals can be stored in the building safety device, thebuilding management system, or safety control panel (collectively, the“system”). Authorized individuals may be able to activate an alarmcondition just by speaking when the system verifies their voice againstthe stored voice samples or patterns 313 a-313 n. For example, if thesystem detects the words “activate fire alarm,” it may first determineif the speaker is an authorized individual. If the speaker is anauthorized individual, the system may activate the alarm condition basedon the spoken instructions alone. If the speaker is not an authorizedindividual, the system may instead use audio or visual indicators todirect the individual to the nearest pull station where the alarmcondition can be manually activated.

In some embodiments, the building safety device 300 can wirelesslycommunicate with a mobile device, such as an individual's mobile phonevia a secondary communication interface 314, using Bluetooth, Wi-Fi, orother wireless protocols. In such embodiments, the mobile device mayperform speech recognition functions and communicate the result to thebuilding safety device. For example, a visually-impaired user mayactivate an application (“App”) on their mobile device requesting thelocation of the nearest pull station. The App communicates this requestfrom the mobile device to the building safety device, and the systemresponds by directing the individual to the nearest pull station.

In some embodiments, the system can automatically identify the physicallocation of an individual using infrared detectors, the microphone,wireless technology that identifies the individual's mobile device, orotherwise, such as by location sensors/devices 330 as disclosed herein.Automatically identifying the physical location of the individual helpsenable the system to provide directions to the individual.

In such cases, the system can provide audible directions, using thespeaker 306 of the applicable building safety device 300 or otherspeaker under the control of the system (i.e., public speaker 131 or aspeaker employed in any notification device 112), to guide theindividual to the pull station, exit, or other location the individualis seeking. In cases where the system can actively track the location ofthe user, the system can also give audible feedback to the individualsuch as “you are moving in the right direction” or “you are going thewrong way.” The system can store data such as building plans, buildingplanning systems, and locations within the building of exits, pullstations, and other destinations to automatically provide direction toindividuals.

In some embodiments, when an alarm condition is activated and the systemknows the location of an individual in the building, the system canalert emergency personnel of the individual's location so thatassistance can be provided.

In some embodiments, the audible notification can include a high pitchsound and/or low soft tone to aid service animals to locate pullstations or other destinations.

The building safety device 300 can be implemented in any number of firesafety devices and/or other building devices common to buildings, suchas pull stations, exit signs, fire/smoke/heat detectors, strobe lights,thermostats or other building controls, or otherwise, to leverage theexisting infrastructure of the building and to provide the capabilitiesdescribed herein to every part of the building.

FIG. 4 depicts a flowchart of a process 400 in accordance with disclosedembodiments. This process may be performed, for example, by one or morebuilding safety systems, such as building safety system 100, one or morebuilding safety devices such as building safety device 300, or one ormore data processing systems, such as, for example, the data processingsystem 200, configured to perform acts described below, a mobile device120, any of which, or any combination of which, are referred to in thesingular as “the system.” The process may be implemented by executableinstructions stored in a non-transitory computer-readable medium thatcause one or more elements of the system to perform such a process. Forexample, the emergency safety application 228 may comprise theexecutable instructions to cause one or more data processing systems toperform such a process.

The process begins with the system detecting a voice input from anindividual (405). In various embodiments, the voice input is received bythe microphone 304 with interface with the voice recognition circuit(331) of a building safety device 300. In other embodiments, the voiceinput can be received by the individual's mobile device. The voice inputcan be, for example, a request for directions to a pull station, acommand to activate an alarm, a sound recognizable by the voicerecognition circuit 331 as a distress sound from an individual (such asa scream), a word recognizable by the voice recognition circuit 331 as adistress word from an individual (such as the word “Fire” or “Help”) inany known languages, or otherwise.

The system receives voice data produced by a speech recognition processperformed on the voice input (410), such as by performing aspeech-recognition process on the voice input to produce the voice data.In various embodiments, the SR process is performed by the safetycontrol panel, the building management system, the building safetydevice, or the individual's mobile device, which includes transmittingthe voice input between devices as necessary, so that the buildingmanagement system receives the voice data. For example, in oneembodiment, the voice recognition circuit 331 of the building safetydevice 300 having the microphone 304 that receives the voice input froman individual within the predetermined distance of the building safetydevice 300 next performs the speech recognition or SR process on thereceived voice input to produce corresponding voice data for furtherprocessing locally at the building safety device 300 or transmittal tothe safety control panel or the building management system for furtherprocessing. When performing the SR process on the received voice input,the system my detect a distress sound by comparing the voice dataproduced from the voice input to a known user voice pattern (e.g.,pattern 313 a) that the system recognizes as a “scream” pattern from anindividual. Similarly, when performing the SR process on the receivedvoice input, the system may detect a distress word by comparing thevoice data to a library of known distress words or patterns 313corresponding to known distress words in known languages that are storedin the safety control panel, the building management system, orapplicable building safety device that received the voice input, such as“help”, “fire”, “smoke”, “earthquake”, “terrorist,” “attack”, etc.

The system can optionally determine whether the voice data correspondsto an authorized user (415). Again, this determination can be performedby the safety control panel, the building management system, thebuilding safety device, or the individual's mobile device, whichincludes transmitting the voice data between devices as necessary. Thisdetermination can be performed, for example, by comparing the voiceinput or voice data to stored voice samples or patterns or stored voicedata of authorized users or predetermined keywords. In one embodiment,user voice samples or patterns 313 a-313 n may be received by the systemfrom respective individuals that have used an application on theirmobile device 120 (e.g., such as Apple's Siri voice activated app) tocapture their voice pattern and submit their captured voice pattern tothe system (e.g., to the fire control panel or management monitoringsystem hosting the emergency safety application 228, or the secondarycommunication interface 314 of the applicable building safety device300).

The system can determine the location of the individual (420). Using anyof the techniques described herein, the system determines the locationof the individual in the building or relative to the nearest buildingsafety device. For example, if the person needs to reach a pull stationor exit, the system can use sensors 330 to derive the location of theuser relative to the building safety device and then provide an audibleresponse to guide the user to the device or exit as described below.

The system determines a response to the voice input (425). The responsecan include an audible response produced by the system, a visualresponse produced by the system, activation of an alarm condition,contacting emergency personnel, or some combination of these.Determining the response can be based on the determination of whetherthe voice sample, pattern, or data corresponds to an authorized user, asthe response can differ depending on if it was made by an authorizeduser. Determining the response is based on the voice data; that is, therecognized speech can be a request for directions to a pull station, acommand to sound an alarm, a distress sound, a distress word, orotherwise.

In one example, the response is an audible response, by a buildingsafety device, directing the individual to a pull station. In anotherexample, particularly where the individual is an authorized user, theresponse can include activating an alarm condition, notifying emergencypersonnel, and performing other functions that might otherwise require aphysical activation of a manual pull station.

In embodiments where the voice input is received by the mobile device,the voice input and voice data are transmitted by the mobile device tothe remainder of the building safety system for the response to beproduced. Note that the form of the voice input may or may not bespecifically required; that is, in some embodiments, specific key wordsor commands may be required, while in other cases the voice input isparsed to determine its nature, e.g., if it is a request for thelocation of a pull station, a request to sound an alarm, or otherwise.

In embodiments where an initial voice input and corresponding voice datais recognized by the system as a distress sound (such a “scream”) or adistress word (such a “Fire”) from an authorized user, the system maydetermine that there is an emergency event and immediately produce aresponse, such as activating an alarm condition in the building and/ornotifying emergency personnel.

In other embodiments, the system recognizes the initial voice input andcorresponding voice data as a distress sound or as a distress word froman unauthorized user. In these embodiments, the system recognizes thatthe initial voice input/data does not correspond to a voice sample orpattern stored by the system to determine that the distress sound or thedistress word is not from an authorized user. In such embodiments, thesystem may provide a verbal response via a speaker 306 of the buildingsafety device 300 closest to the location of the user or individual thatprovided the initial voice input. The verbal response may be a phrase toprompt the user or individual to confirm an emergency event isoccurring. (e.g., “Do you need to activate fire alarms in your area?” or“Do you need emergency personnel at your current location?” or “Do youneed assistance in locating an emergency exit?” The system maysubsequently wait for a second voice input at 405 before continuingprocessing, as indicated by loop 435.

The system may then produce the response to the voice input (430). Insome embodiments, this action or step by the system can includetransmitting a response to the mobile device so that the mobile devicepresents the response to the individual. The process can return todetecting voice inputs at 405, as indicated by loop 440.

FIG. 5 depicts a flowchart of a process 500 in accordance with disclosedembodiments. This process may be performed, for example, by one or morebuilding safety systems, such as building safety system 100, one or morebuilding safety devices such as building safety device 300, or one ormore data processing systems, such as, for example, the data processingsystem 200, configured to perform acts described below, a mobile device,any of which, or any combination of which, are referred to in thesingular as “the system.” The process may be implemented by executableinstructions stored in a non-transitory computer-readable medium thatcause one or more elements of the system to perform such a process. Forexample, the emergency safety application 228 may comprise theexecutable instructions to cause one or more data processing systems toperform such a process.

The process begins with the system detecting a voice input from anindividual (505). In various embodiments, the voice input is received bythe microphone 304 with interface with the voice recognition circuit(331) of a building safety device 300. In other embodiments, the voiceinput can be received by the individual's mobile device. The voice inputcan be, for example, a request for directions to a pull station, acommand to activate an alarm, a distress sound, a distress word, orotherwise.

The system receives voice data produced by a speech recognition processperformed on the voice input (510), such as by performing aspeech-recognition process on the voice input to produce the voice data.In various embodiments, the SR process is performed by the safetycontrol panel, the building management system, the building safetydevice, or the individual's mobile device, which includes transmittingthe voice input between devices as necessary, so that the buildingmanagement system receives the voice data. For example, in oneembodiment, the voice recognition circuit 331 of the building safetydevice 300 having the microphone 304 that receives the voice input froman individual within the predetermined distance of the building safetydevice 300 next performs the speech recognition process on the receivedvoice input to produce corresponding voice data for further processinglocally at the building safety device 300 or transmittal to the safetycontrol panel or the building management system for further processing.

The system can optionally determine whether the voice data correspondsto any access rights level (515). Again, this determination can beperformed by the safety control panel, the building management system,the building safety device, or the individual's mobile device, whichincludes transmitting the voice data between devices as necessary. Thisdetermination can be performed, for example, by comparing the voiceinput or voice data to stored voice samples or patterns or stored voicedata of authorized users or predetermined keywords. An access rightslevel can be for example, a public user, an employee of a facility,emergency personnel, or otherwise. The access rights level can bedetermined, for example, by keywords or passphrases (or lack of them) inthe voice input or voice data.

The system determines a response to the voice input that is appropriateto the corresponding access rights level or situation (520). Theresponse can include an audible response produced by the system, avisual response produced by the system, activation of an alarmcondition, contacting emergency personnel, some combination of these, orotherwise. Determining the response is based on the voice data; that is,the recognized speech can be a request for directions to a pull station,a command to sound an alarm, or otherwise. In one example, the responseis an audible response, by a building safety device, directing theindividual to a pull station. The response appropriate to an accessright level can include ignoring the voice input (for example, if apublic user says “fire” but does not have rights to activate a firealarm by speaking, directing the user to a pull station (for example, ifa public user says “fire alarm” or “pull station”), or otherwise. Inanother example, particularly where the individual has a “higher” accessrights level, the response can include activating an alarm condition,notifying emergency personnel, and performing other functions that mightotherwise require a physical activation of a manual pull station. Inembodiments where the voice input is received by the mobile device, thevoice input and voice data are transmitted by the mobile device to theremainder of the building safety system for the response to be produced.Note that the form of the voice input may or may not be specificallyrequired; that is, in some embodiments, specific key words or commandsmay be required, while in other cases the voice input is parsed todetermine its nature, e.g., if it is a request for the location of apull station, a request to sound an alarm, or otherwise.

The system produces the response to the voice input (525). In someembodiments, this can include transmitting a response to the mobiledevice so that the mobile device presents the responds to theindividual. The response can be an audible response as described herein,such as directions to the nearest pull station or exit. In some cases,the system can also temporarily silence other alarms in the vicinity sothat a visually-impaired individual can hear the audible response anddirections, and reactivate the other alarms when the visually-impairedindividual has exited the area or reached the pull station. The processcan return to detecting voice inputs at 505, as indicated by loop 540.

The system can detect a manual activation of a pull station (530), andproduce an appropriate response such as activating an emergencycondition, sounding an alarm, notifying authorities, or otherwise.Again, the process can return to detecting voice inputs at 505, asindicated by loop 545.

Disclosed embodiments provide distinct technical advantages over presentsystems. In particular, disclosed embodiments enable the visuallyimpaired to speak their need for a pull station or to sound an alarm,and the building safety system automatically responds appropriately,sounding an alarm, directing the individual to a pull station or exit,or other audible or visual response.

Those skilled in the art will recognize that, for simplicity andclarity, the full structure and operation of all data processing systemssuitable for use with the present disclosure is not being depicted ordescribed herein. Instead, only so much of a data processing system asis unique to the present disclosure or necessary for an understanding ofthe present disclosure is depicted and described. The remainder of theconstruction and operation of data processing system 200 may conform toany of the various current implementations and practices known in theart.

It is important to note that while the disclosure includes a descriptionin the context of a fully functional system, those skilled in the artwill appreciate that at least portions of the mechanism of the presentdisclosure are capable of being distributed in the form of instructionscontained within a machine-usable, computer-usable, or computer-readablemedium in any of a variety of forms, and that the present disclosureapplies equally regardless of the particular type of instruction orsignal bearing medium or storage medium utilized to actually carry outthe distribution. Examples of machine usable/readable or computerusable/readable mediums include: nonvolatile, hard-coded type mediumssuch as read only memories (ROMs) or erasable, electrically programmableread only memories (EEPROMs), and user-recordable type mediums such asfloppy disks, hard disk drives and compact disk read only memories(CD-ROMs) or digital versatile disks (DVDs).

Although an exemplary embodiment of the present disclosure has beendescribed in detail, those skilled in the art will understand thatvarious changes, substitutions, variations, and improvements disclosedherein may be made without departing from the spirit and scope of thedisclosure in its broadest form.

None of the description in the present application should be read asimplying that any particular element, step, or function is an essentialelement which must be included in the claim scope: the scope of patentedsubject matter is defined only by the allowed claims. Moreover, none ofthese claims are intended to invoke paragraph six of 35 USC § 112 unlessthe exact words “means for” are followed by a participle.

What is claimed is:
 1. A method performed by a building safety systemhaving at least a processor, the method comprising: by the buildingsafety system: receiving a voice data relating to a noise produced froman individual, the voice data produced by a speech recognition processperformed on the noise, wherein the noise is a first voice input;determining whether the noise corresponds to an authorized individual;determining a presence and direction of movement of the individual;activating an alarm condition and producing a directional response basedon the noise in response to determining that the noise corresponds tothe authorized individual, wherein the directional response is based onthe presence and direction of movement of the individual, effective todirect the individual to a further location; prompting the individual toconfirm that an emergency event is occurring in response to determiningthat the noise does not correspond to any authorized individual;receiving a second voice input in response to prompting the individual;and producing a directional response based on the second voice input inresponse to receiving the second voice input.
 2. The method of claim 1,wherein the building safety system comprises a location sensor effectivefor the determining the presence of the individual and for determining adirection of movement of the individual.
 3. The method of claim 2,wherein the location sensor comprises a device selected from the groupconsisting of infrared detector or array, heat detector or array,camera, motion sensor, microphone, ultrasonic ranging module, a sonicrangefinder, laser rangefinder, radar distance measurement, pulserepetition frequency devices, GPS devices and wireless device.
 4. Themethod of claim 2, comprising the building safety system selecting thefurther location based on the noise.
 5. The method of claim 4, whereinthe building safety system comprises a plurality of building safetydevices each having a speaker, and wherein the directional response isan audible response via the speaker comprised of a first of theplurality of building safety devices.
 6. The method of claim 2, whereinthe further location is an exit.
 7. The method of claim 6, comprisingwherein the building safety system comprises a storage of plurality ofvoice patterns, and wherein the building safety system determines whenthe noise corresponds to the authorized individual based on the storedvoice patterns.
 8. The method of claim 1, wherein the speech recognitionis performed by a mobile device.
 9. The method of claim 1, wherein thespeech recognition is performed by the building safety system.
 10. Themethod of claim 9, wherein the building safety system has a plurality ofbuilding safety devices, each having a microphone, the noise is receivedby a first of the plurality of building safety devices, and the speechrecognition process is performed by a safety control panel incommunication with the first of the plurality of building safetydevices.
 11. The method of claim 1, determining the noise is a requestfor directions to a pull station, a distress sound, or a distress word.12. The method of claim 11, activating a notification device controlledby the building safety system to signal an alarm in response todetermining the noise is directions to a pull station, a distress sound,or a distress word.
 13. The method of claim 11, wherein the buildingsafety system comprises a plurality of building safety devices eachhaving a speaker, and after determining the noise is a distress sound ora distress word, the building safety system produces an audible promptfor the individual to confirm an emergency event is occurring.
 14. Themethod of claim 1, wherein the noise is received by a building safetydevice, wherein the building safety device is selected from the groupconsisting of pull station, smoke detector, heat detector, notificationdevice, emergency light, exit sign, and public-address speaker.
 15. Themethod of claim 1, wherein the directional response is produced abuilding safety device, wherein the building safety device is selectedfrom the group consisting of pull station, smoke detector, heatdetector, notification device, emergency light, exit sign, andpublic-address speaker.
 16. The method of claim 1, further comprising:silencing at least one alarm in a vicinity of the individual other thanthe directional response; and reactivating the at least one alarm whenthe individual has exited the vicinity or reached a building safetydevice.
 17. A building safety system, comprising: a safety controlpanel; and a plurality of building safety devices in communication withthe safety control panel, the building safety system configured to:receive voice data relating to a noise produced from an individual, thevoice data produced by a speech recognition process performed on thenoise, wherein the noise is a first voice input; determine whether thenoise corresponds to an authorized individual; determine a presence anddirection of movement of the individual; activate an alarm condition andproduce a directional response based on the noise in response todetermining that the noise corresponds to the authorized individual,wherein the directional response is based on the presence and directionof movement of the individual, effective to direct the individual to afurther location; prompt the individual to confirm that an emergencyevent is occurring in response to determining that the noise does notcorrespond to any authorized individual; receive a second voice input inresponse to prompting the individual; and produce a directional responsebased on the second voice input in response to receiving the secondvoice input.
 18. The building safety system of claim 17, each of theplurality of building safety devices comprises a location sensoreffective for the determining the presence of the individual and fordetermining a direction of movement of the individual.
 19. The buildingsafety system of claim 18, wherein the location sensor comprises adevice is selected from the group consisting of infrared detector orarray, heat detector or array, camera, motion sensor, microphone,ultrasonic ranging module, a sonic rangefinder, laser rangefinder, radardistance measurement, pulse repetition frequency devices, GPS devicesand wireless device.
 20. The building safety system of claim 18, whereinthe building safety system comprises a plurality of building safetydevices each having a speaker, and wherein the directional response isan audible response via the speaker included in a first of the pluralityof building safety devices.